multi-modal feature engineering
Predicting Depression Severity by Multi-Modal Feature Engineering and Fusion
Samareh, Aven (University of Washington) | Jin, Yan (University of Washington) | Wang, Zhangyang (Texas A&M University) | Chang, Xiangyu (Xi'an Jiaotong University) | Huang, Shuai (University of Washington)
We present our preliminary work to determine if patient's vocal acoustic, linguistic, and facial patterns could predict clinical ratings of depression severity, namely Patient Health Questionnaire depression scale (PHQ-8). We proposed a multi-modal fusion model that combines three different modalities: audio, video, and text features. By training over the AVEC2017 dataset, our proposed model outperforms each single-modality prediction model, and surpasses the dataset baseline with a nice margin.